An agent-based reinforcement learning approach to improve human-robot-interaction in manufacturing

This work is aimed at the understanding and application of several emerging technologies as they relate to improving the interactions which occur between robotic operators and their human colleagues across a range of manufacturing processes. These interactions are problematic, as variation in performance of human beings remains one of the largest sources of disturbances within such systems, with potentially significant implications for productivity if it continues unmitigated. The problem remains for the most part unaddressed, despite these interactions becoming increasingly prevalent as the rate of adoption of automation technologies increases. By reconciling multiple areas encompassed by the wider domain of intelligent manufacturing, the presented work identifies a methodology and a set of software tools which leverage the strengths of neural-network-based reinforcement learning to develop intelligent software agents capable of adaptable behaviour in response to observed environmental changes. The methodology further focuses on developing representative simulation models for these interactions following a pattern of generalisation, to effectively represent both human and robotic elements, and facilitate implementation. By learning through their interaction with the simulated manufacturing environment, these agents can determine an appropriate policy, by which to autonomously adjust their operating parameters, as a response to changes in their human colleagues. This adaptability is demonstrated to enable the intelligent agents to determine an action policy which results in less observed idle time, along with improved leanness and overall productivity, over multiple scenarios. The findings of the work suggest that software agents that make use of a reinforcement based learning approach are well suited to the task of enabling robotic adaptability in such a way, and the developed methodology provides a platform for further development and exploration, along with numerous insights into the effective development of these agents

A framework of integrating knowledge of human factors to facilitate HMI and collaboration in intelligent manufacturing

Recent developments in the field of intelligent manufacturing have led to increased levels of automation and robotic operators becoming commonplace within manufacturing processes. However, the human component of such systems remains prevalent, resulting in significant disturbance and uncertainty. Consequently, semi-automated processes are difficult to optimise. This paper studies the relationships between robotic and human operators to develop the understanding of how the human influence affects these production processes, and proposes a framework to integrate and implement knowledge of such factors, with the aim of improving Human-Machine-Interaction, facilitating bi-directional collaboration, and increasing productivity and quality, supported by an example case-study

Improving human-robot-interaction utilizing learning and intelligence: a human factors-based approach

Several decades of development in the fields of robotics and automation have resulted in human-robot interaction is commonplace, and the subject of intense study. These interactions are particularly prevalent in manufacturing, where human operators (HOs) have been employed in numerous robotics and automation tasks. The presence of HOs continues to be a source of uncertainty in such systems, despite the study of human factors, in an attempt to better understand these variations in performance. Concurrent developments in intelligent manufacturing present opportunities for adaptability within robotic control. This article examines relevant human factors and develops a framework for integrating the necessary elements of intelligent control and data processing to provide appropriate adaptability to robotic elements, consequently improving collaborative interaction with human colleagues. A neural network-based learning approach is used to predict the influence on human task performance and use these predictions to make informed changes to programed behavior, and a methodology developed to explore the application of learning techniques to this area further. This article is supported by an example case study, in which a simulation model is used to explore the application of the developed system, and its performance in a real-world production scenario. The simulation results reveal that adaptability can be realized with some relatively simple techniques and models if applied in the right manner and that such adaptability is helpful to tackle the issue of performance disparity in manufacturing operations

The ethical use of human data for smart manufacturing: an analysis and discussion

Much of the recent work in the field of smart manufacturing is dependent on a data-driven approach, with an increasing number of data-hungry techniques being introduced to improve the adaptability and productivity of manufacturing systems. More recently, the collection and use of data generated by employees and individuals has become the source of controversy at both a corporate and societal level. This work presents an analysis of current thoughts and considerations on the use and misuse of data from a number of standpoints and discusses a methodology that enables appropriate parameterization of human performance for use in modelling and simulation for smart manufacturing

Integrating intelligence and knowledge of human factors to facilitate collaboration in manufacturing

The implementation of automation has become a common occurrence in recent years, and automated robotic systems are actively used in many manufacturing processes. However, fully automated manufacturing systems are far less common, and human operators remain prevalent. The resulting scenario is one where human and robotic operators work in close proximity, and directly affect the behavior of one another. Conversely to their robotic counterparts, human beings do not share the same level of repeatability or accuracy, and as such can be a source of uncertainty in such processes. Concurrently, the emergence of intelligent manufacturing has presented opportunities for adaptability within robotic control. This work examines relevant human factors and develops a learning model to examine how to utilize this knowledge and provide appropriate adaptability to robotic elements, with the intention of improving collaborative interaction with human colleagues, and optimized performance. The work is supported by an example case-study, which explores the application of such a control system, and its performance in a real-world production scenario

Reinforcement learning for facilitating human-robot-interaction in manufacturing

For many contemporary manufacturing processes, autonomous robotic operators have become ubiquitous. Despite this, the number of human operators within these processes remains high, and as a consequence, the number of interactions between humans and robots has increased in this context. This is a problem, as human beings introduce a source of disturbance and unpredictability into these processes in the form of performance variation. Despite the natural human aptitude for flexibility, their presence remains a source of disturbance within the system and make modelling and optimization of these systems considerably more challenging, and in many cases impossible. Improving the ability of robotic operators to adapt their behaviour to variations in human task performance is, therefore, a significant challenge to be overcome to enable many ideas in the larger intelligent manufacturing paradigm to be realised. This work presents the development of a methodology to effectively model these systems and a reinforcement learning agent capable of autonomous decision-making. This decision-making provides the robotic operators with greater adaptability, by enabling its behaviour to change based on observed information, both of its environment and human colleagues. The work extends theoretical knowledge on how learning methods can be implemented for robotic control, and how the capabilities that they enable may be leveraged to improve the interaction between robots and their human counterparts. The work further presents a novel methodology for the implementation of a reinforcement learning-based intelligent agent which enables a change in behavioural policy in robotic operators in response to performance variation in their human colleagues. The development and evaluation are supported by a generalized simulation model, which is parameterized to enable appropriate variation in human performance. The evaluation demonstrates that the reinforcement agent can effectively learn to make adjustments to its behaviour based on the knowledge extracted from observed information, and balance the task demands to optimise these adjustments